A Flexible, Parallel Generator of Natural Language

My Ph.D. thesis (Ward 1992, 1991)1 addressed the task of generating natural language utterances. It was motivated by two difficulties in scaling up existing generators. Current generators only accept input that are relatively poor in information, such as feature structures or lists of propositions; they are unable to deal with input rich in information, as one might expect from, for example, an expert system with a complete model of its domain or a natural language understander with good inference ability. Current generators also have a very restricted knowledge of language— indeed, they succeed largely because they have few syntactic or lexical options available (McDonald 1987)— and they are unable to cope with more knowledge because they deal with interactions among the various possible choices only as special cases. To address these and other issues, I built a system called FIG (flexible incremental generator). FIG is based on a single associative network that encodes lexical knowledge, syntactic knowledge, and world knowledge. Computation is done by spreading activation across the network, supplemented with a small amount of symbolic processing. Thus, FIG is a spreading activation or structured connectionist system (Feldman et al. 1988). In the initial state, some nodes representing concepts are sources of activation; this pattern of activation represents the information to be expressed. Activation flows from these nodes to nodes representing words through the various knowledge structures of the network. When the network settles, the most highly activated word is selected and emitted. Activation levels are then updated to represent the new current state, both in syntactic and semantic aspects. This process of settle, emit, and update repeats until all the input has the subject-predicate construction because it is a verb. FIG’s syntactic coverage is much broader than that of previous connectionist generators such as Gasser (1988); output include “once upon a time there lived an old man and an old woman,” “one day the old woman went to a stream to wash clothes,” and “John ate a peach with an old woman’s fork.” The success of this model in generating utterances of English and Japanese suggests that the complexity present in most treatments of syntax is unnecessary: FIG dispenses with the assembly of syntactic structures, constructions that affect the utterance only by the activation they transmit, directly or indirectly, to words. FIG does without a mechanism for explicit syntactic choice; any number of constructions are potentially active, competing or cooperating in parallel, and the choice among them is emergent. Phenomena traditionally considered to require instantiation and variable binding are handled in FIG with much simpler mechanisms. Grammatical output results not from constraints on the form of syntactic structures or the behavior of an algorithm but, rather, from the structure and weights of the network as a whole. This paragraph summarizes the ways in which FIG addresses the issues that motivated its construction: It handles arbitrarily rich input because the number of nodes activated in the initial state makes no difference to its operation. It handles interaction among choices easily because it tends to settle into a state representing a compatible set of choices as a result of links among nodes that represent such choices. It handles trade-offs among competing goals without additional mechanism because all computation is in terms of numbers. Thus, FIG is the first generator potentially able to perform well at the complex generation tasks that will arise in the future. Of course, to realize this potential requires more experimentation with the details of activation flow and with ways to A Flexible, Parallel Generator of Natural Language